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(57) Abstract 



The current frame of the picture is com- 
pared (2) block-by-block with the previous frame 
to identify changed picture elements (pixels). The 
resulting matrix Is matched ("vector quantised") 
(4) to one of a set of standard matrices ("VQ 
shapes"). Only those pixels flagged by the select- 
ed YQ shape are transmitted plus a "VQ shape 
code". This technique is modified in that the cur- 
rent frame is also compared (30) with a "back- 
ground" frame stored (31) at the transmitter and 
at the receiver. Where, for any block, ail the pix- 
els flagged by the VQ shape are indicated by the 
second comparison as being the same as the 
background, the pixels are not sent - instead a 
shape code plus a "copy background" instruction 
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5IGNRL CODING 

The present invention relates to coding of video 
signals, especially using conditional replenishment 
coding, where information is generally transmitted only in 
resnect of elements of a flan* of the picture which have 
changed relative to a previous frame; the transmitted data 
being used at a receiver to update a stored version of the 
picture. 

Thus picture elements of each block of an image to be 
coded are copied vith those of the corresponding block 
of a previously coded image to determine whether the block 
has changed between the two inages; if so, picture element 
data are generated for output. 

Such a system is described in international patent 
application cashed under no. W086/Q3922, which also 
proposes that the block be compared with the corresponding 
block of an earlier (reference) image. If they are deemes 
to be the sane, no picture data are generated - instead, a 
codeword is produced to indicate that the receiver is to 
obtain its data from a locally stored replica cf the 
reference image. 

Recording to one aspect of the present invention, 
there is provided a method of coding an image comprising, 
for each of a plurality of blocks of an image: 
(i) comparing picture elements of the block with those 
of the corresponding block of a previously coded 
iraage to produce a matrix of values each 
indicating whether the corresponding element is, 
in accordance with a predetermined criterion, 
deemed to have changed between the two images; 
(+i> matching the matrix to one of a predetermined set 
of such matrices each of which identifies a recicr. 



WO 89/04101 



- 2 - 



PCT/GB88/0OS71 



of the block as being deemed to have changed, and 
generating a codeword identifying that one matrix 
of the eetf 

(iii) comparing picture . elements of the block with those 
of the corresponding block, of a reference image 
composed of blocks- from at least one earlier frame 
to determine whether the block is, in accordance 
with a predetermined criterion, deemed to have 
changed between the two images? 

(iv) coding for output those elements within, the 
identified .region, unless all the elements within 
the region are identified by comparison step (iii) 
as being unchanged relative to the reference 
image, whereupon a codeword indicating this is 
generated. 

in another aspect, the invention provides an apparatus 
for coding an image comprising* 

U) means for comparing, for each of a plurality of 
blocks of an image, picture elements of the block 
with those of the corresponding block of a 
previously coded image to produce a matrix of 
values each .indicating, whether the corresponding 
element is, in accordance with a predetermined 
criterion, deemed to have changed between the two 
images; 

(ii) means for matching the matrix to one of a 
predetermined set of such matrices each of which 
identifies a region of the block as being deemed 
to have changed, and generate a codeword 
identifying that one matrix of the set; 

(iii) .a store arranged to store a reference image 

composed of blocks from st least cue earlier frame 
and means for comparing picture elements of the 
block with those of the corresponding block of the 
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reference image to determine whether the block is, 
in accordance with a predetermined criterion, 
deemed to have changed between the two images; 
(iv) means for coding for output those elements within 
the identified region, unless all the elements 
within the region are identified by comparison 
step (iii) as being unchanged relative to the 
reference image, whereupon a codeword indicating 
this is generated. 
T»e matching step - often termed vector quantisation - 
of two-dimensional maps has been proposed previously for 
picture coding, for example in European patent application 
serial no. C239076&, where blocks of transform coefficient 
blocks are classified. 

The present invention, however, further exploits the 
vector quantisation, in that not only =an tie 
classification be used (if desired) to reduce the «n.. 
of information that has to be transmitted to mdica.e 
which elements have been coded for output, but also th 
nunfcer of occasions on which a "reference" codewor* is 
instead generated are increased. 

some embodiments of the invention will now be 
describe*, by way of example, with reference to tae 
accompanying drawings, in which: 

- Figure 1 is a block diagram of a coder accorcung to 
one embodiment of the invention; 

- Figure 2 shows a typical bit map produced by the 
chanae detector of the coder of Figure 1? 

1 Figure 3 illustrates a few standard tit patterns 
used by the vector quantiser of the coder of Figure 1; and 

- Figure 4 is a block diagram of a decoder according 
to another embodiment of the present invention. 

Figure 1 shows a conditional replenished viaeo 
code^ where video signals (assumed to be in digital form) 



are supplied to an input 1. The current frame of a 
picture is compared in a change or movement detector 2 
with the output of a local decoder 3. The local decoder 
produces a "previous frame" output which is the same as 
that produced by a remote decoder; the object of the 
comparison- is to identify those parts of the picture which 
have changed so that only information concerning those 
parts needs to be sent to the decoder, to update a stored 
representation of the frame. 

The signals are processed on a block-by-block basis - 
an 8 x . 8 block is assumed - and the change detector 2 
therefore produces an 8 x 8 bit map. A typical map is 
shown in figure 2, where the elements of the block 
corresponding to picture elements (pixels) which have 
changed are shown shaded. 

In practice, transmission of information concerning 
onl y the changed elements involves a significant 
addressing overhead and therefore it is preferred to ma-ch 
the bit map to one of a limited number - typically forty - 
of standard shapes (a few are illustrated in figure 3). 
Since it is preferable to transmit information for an 
unchanged pixel than tc fail to transmit information for a 
changed pixel, the shape chosen is the smallest {i.e. 
with the least number of shaded elements) which has a 
shaded area covering the shaded elements of the bit map. 
This process is termed vector quantisation (VQ) and is 
indicated as vector quantizer 4 in figure l. One 
transmits, for the block, a VQ number identifying the 
chosen shape, along with information concerning pixels 
deemed - in accordance with the chosen shape - tc have 
changed. The output data are combines ana buffered in a 
buffer 5 prior to passage to an output 6. is is 
conventional in such systems, the buffer is used to smooth 
variations in the rate at which data are generated (cue to 
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the picture content dependent coding) and interface to a 
regular transmitted rate and the buffer fullness state 
used to control the rate of generation (e.g. by varying 
the change detector thresholds (control line 7)). 
s The pixels deemed to have changed are coded in an 

encoder 8 (for example in the manner described in our 
international patent application PCT/GB88/00709 

{publication no ) and European patent 

application no. 88307981.6 (publication no ). 

io Many televised scenes - especially in a 

videoccnference or videotelephone environment - contain 
moving persons or objects set against a fixed background. 
Pixels observed as changed by the charge detector 2 will 
relate either to objects which have changed their position 
15 (or entered the scene) or to parts of the background 

uncovered by the object, in the present coder, a second 
change detector 30 is also shown, which compares the 
current frame with a referer.ce or background frame stored 
in a frame store 31. Acquisition of the stored background 
2D frame will be discussed further below, but in figure 1 is 

assumed to have been acquired from the local decoder 3 in 
response to manual operation of a switch 22 a; the 
commencement of a transmission; a code being transmitted 
to the receiver to initiate similar action si the remote 
25 decoder. 

The change detector 30 produces a bit map identifying 
those pixels of the current frame which are the same as 
the background. 

If this shows that the new image is different from the 
30 background for any of the pixels declared as changed by 

the VQ shape the background information car.r.ot be used and 
information concerning those pixels is transmitted along 
with the VQ number. 
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If, however the changed area as given by the VQ shape 
covers only pixels which are identified by the detector 30 
as being the same as the corresponding pixels of the 
stored background frame, then the VQ number is accompanied 
5 by a reserved 'codeword indicating "background" and no 

further information needs to be transmitted for the block 
in question. 

These functions are accomplished in figure 2 by a 
comparator 33 whose output 34 overrides the operation of 
10 the encoder 8. 

Figure 4 shows a decoder. an input processor 40 
receives the. coded input Eignais. The background frame is 
stored in a background store 41. In normal operation, the 
processor 40 uses the pixel information received to update 

15 via line 42 the contents of a frame store 43, using the 

received VQ numbers via line 44 to control the frarae store 
addressing. When, however, it receives the reserved 
"background" codeword (via line 45), it recovers the 
relevant pixels (identified by the \ r Q number) from the 

20 background store 41 and enters then into the frame store 

43. This is illustrated schematically by a changeover 
switch 45. The frame store 43 is read cut (by output 
centre! means 47) to produce the received video at an 
output 46. As in the coder, the background store 41 is 

25 loaded from the decoded image in store 43 when a 'load 

background' instruction is received (line 49). 

Hote that the- local decoder 3 of Figure 1 can be of 
the same construction as the decoder of Figure 4, 
although, of course, in practice the local decoder would 

30 use the background store 31 rather tfcar. contain one of its 
own. 

Hie change detectors 2,30 can in principle be any 
conventionally used, or may both be as described in our 
above-mentioned patent applications. The vector 
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quantiser 4 can again be a known device, one possibility 
is that described in our international patent application 
no. PCT/SB87/00816 (publication no. W08B/04084) and 
European patent application no. 8527787 {publication no. 

5 0272794), 

Simpler but cruder, another option is to convert the 
BxB matrix to a 4x4 matrix by creating each 'new' element 
as an OR function of four 'oia 1 elements; this reduces the 
number of elements in the matrix to a size (IS) which can 

io be used to address a look-up table - in the form of a 64 

Kbyte read only memory in which the appropriate VQ numbers 
are stored. 

The background scene may be 'frozen' manually at the 
commencement of s transmission. It may, but does not have 
is to, consist solely of a fixed background. For example it 

may include seated figures (thereby covering the situation 
where a person momentarily passes his hand over His face 
if the face forms pert of the "background", the face does 
not have to be retransmitted). 
20 it may be desirable to include provision for updating 

the background. For example, the incoming video could be 
monitored at the coder (by a unit 35 shown dotted in 
Figure 1} and parts of the picture which differ from the 
original background but have remained unchanged for a 
25 predetermined period of time inserted into the frame 

store, a signal being sent (via line 36) to the decoder to 
instruct it to do likewise. 

An alternative method of updating the background store 
abandons any attempt at identifying genuine background, 
30 but instead forms a reference image which is a composite 
of blocks taker, from preceding images over a period. Each 
frame period, data for a few (e.g. eight) selected blocks 
scattered over the image area is er.tered into the 
background stores, the block selection being such that 
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different blocks are selected from each frame until the 
whole image area has been covered. Assuming 1024 blocks 
per frame at 25 frames per second, this represents a 
period of approximately 40 seconds. The blocks could be 
loaded from the frame store 43 of the decoder, and a 
corresponding store in the local decoder 3, the change 
monitor then being replaced by a simple address generator 
35 to select the appropriate blocks. Clearly, this is 
less effective, in that some blocks of the reference image 
will not represent background material, but a significant 
coding advantage is still obtained, and it has the merit 
of simplicity. 

A variation of this approach, provides that data for 
selected blocks, instead of being drawn from the decoded 
image, are actually transmitted - i.e. transmission of the 
whole block is forced even if only a part, or none, of it 
is indicated by the detector 2 and quantiser 4 as moving. 
In this case, the frame stares 31, 41 take their input 
from the input 1 and processor 40 respectively. This else 
has the benefit of ensuring that transmission errors on 
blocks which rarely change do not persist ir. -the decoded 
image . 
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1. a method of coding an image comprising, for each 
of a plurality of mocks of an image: 

(i) comparing picture elements of the block with 
those of the corresponding block of a previously 
coded image to produce a matrix of values each 
indicating whether the corresponding element is, 
in accordance with a predetermined criterion, 
deemed to have changed between the two images; 

(ii) matching the matrix to one of a predetermined set 
of such matrices each of which identifies a 
region of the block as being deemed to have 
changed, and generating a codeword identifying 
that one matrix of the set? 

(iii) comparing picture elements of the block with 
those of the corresponding block of a reference 
image composed of blocks from at least one 
earlier frame to determine whether the block is, 
in accordance with a predetermined criterion, 
deemed to have changed between the two images; 

(iv) coding for output those elements within the 
identified region, unless ail the elements within 
the region are identified by comparison step 
(iii) as being unchanged relative to the 
reference image, whereupon a codeword indicating 
this is generated. 

2. A method according to claim 1 in which, in the 

coding of the- elements within the identified region, at 
least some of the elements are coded as the difference 
between that element and a predicted value for that 
element derives from one or more previously coded elements 
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of the block, the sequence of coding of the elements being 
dependent on the orientation of the identified region. 

3. An apparatus for coding an image comprising, 

(i) means for comparing, for each of a plurality of 
blocks of an image, picture elements of the block 
with those of the corresponding block of a 
previously coded image to produce a matrix of 
values each indicating whether the corresponding 
element is, in accordance with a predetermined 
criterion, deemed to have changed between the two 
images; 

£ii) means for matching the matrix to one of a 
predetermined set of such matrices each of which 
identifies a region of the block as being deemed 
to have changed, and generate a codeword 
identifying that one matrix of the set? 

(iii) a store arranged to store a reference image 
composed of blocks from st least one earlier 
frame and means for comparing picture elements of 
the block with those of the corresponding block 
cf the reference image -o . determine whether the 
blocfc is, in accordance with a predetermined 
criterion, deemed to have changed between the two 
images; 

Civ) means for coding for output those elements within 
the identified region, unless all the elements 
within the region are identified by comparison 
step (iii) as being unchanged relative to ' the 
reference image, whereupon a codeword indicating 
this is generated. 

4. An apparatus according to claim 3, including 

means for comparing successive images to identify pariis cf 
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the image which have remained unchanged for a 
predetermined period of tine, and in the event of such 
parts being identified, to: 

(a) update the stored reference linage; ana 

(b) code for output data indicating which parts of 
the reference image have been thus updated. 

5. ftn apparatus according to claim 3, including 

means arranged In operation to effect periodic replacement 
of a minority of the blocks of the reference image stored 
in the store by the corresponding blocks of a recent 
image, the blocks being differently selected for 
successive replacements such that the store always 
contains a composite image made up of blocks from a 
plurality of frames of the image being coded. 

6. ft decoder for use with the coder Df claim 3 or 5, 

comprising; 
(a) 



a frame store for storing a received image 
(b) a second frame store 

to 



control means responsive to ' received data to 
update the contents of the second frame store 
using the received data and responsive to a 
received codeword to update the second frame 
store with information from the first frame store? 
means for repetitively reading the contents of 
the second frame store to produce a video output 
signal. 
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